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was found that certain areas of linguistic response warranted serious 
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ABSTRACT 



The purpose of the study was to locate measurable 
criteria of responsiveness to language which have not been 
utilized to any significant extent in standardized tests. 

The project involved creating a testing instrument, reviewing 
of the instrument by a panel of experts, and applying the 
resultant test to a small population of college students and 
to a criterion group of acknowledged sensitive users of English. 
Comparison was made between the performances of students and 
members of the criterion group by means of a pre-and-post test. 
Reliability and validity studies were made of the results. Though 
the test was necessarily crude, it was found that certain areas 
of- linguistic response warranted serious consideration in the 
development of future language competency tests* sociolinguistic 
and semantic context, deep structure of word-groups and sentences, 
primary factors of extended discourse, and, possibly, organization 
of extended discourse. 




ERIC 




INTRODUCTION 



Under a contract awarded by Region X of the U.S. 

Office of Education, Department of Health, Education, and 
Welfare members of the English Department of Portland state 
University, Portland, Oregon, conducted a pilot research 
project to isolate measurable criteria of responsiveness 
to language irrespective of dialect. Research was confined 
to written English in a variety of forms. The project 
was conceived because the researchers felt that most stand- 
ardized language competency tests used in schools and 
colleges were superficial, that they measured the ability 
of the student to know and use certain expected forms of 
standard English, but seldom reached other linguistic abili- 
ties. Responsiveness — the capacity to recognize, if not 
intellectualize, nuances of word meaning, of structural 
form, and of rhetorical organization--was the term we used 
to represent those linguistic and sociolinguistic capacities 
left largely unmeasured by existing tests. Though the term 
was not entirely satisfactory, it did indicate the complexity 
and multiplicity of individual responses to various dialects 
in a speech community, e.g., the person who does not respond 
fully to the dialects of the university or business world 
but may react with great sensitivity to the dialect of his 
own group. To what extent this multiplicity of response was 
due to environmental factors we could not know, for while 
studies in the language of the disadvantaged child emphasize 
his inability to handle complex linguistic forms (Bernstein 
1961, Deutsch 1963), few have probed deeply enough to deter- 
mine whether this inability is merely symptomatic of a parti- 
cular sociocultural environment (Labov 1969a, Wolfram 1969} • 
Regardless of the ultimate causes of inadequacies with some 
particular linguistic forms, specialists generally recognize 
that the formal requirements of "good English" do not really 
reflect native capacity to understand and use another dialect 
of English (Shuy 1964, Labov 1969b, Baratz and shuy 1969, 
Burling 1970). At the same time, no one has adequately 
determined, much less measured, the constituents of this 
native capacity. 

measurement of language competence is difficult) it is 
even more difficult when it attempts to quantify the elements 
of an intuitive ability which amounts to sensitivity to a 
number of complex linguistic factors. There has been some 
research on developing instruments to register linguistic 
aptitude (Carroll 195^» l96l, Hunt 1968, Tikofsky 1968), 
but evaluations of objective tests which attempt to measure 
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"aptitude," "competency," "communicational skill," etc. 
show that the tests are all but inadequate. In a survey 
of evaluations of sixteen standard tests of linguistic 
ability in junior high and high school the researcher con- 
cludes, "the evidence of this study points overwhelmingly 
to the fact that there is no perfect objective test of 
English, nor does any currently published test come close 
to the goal of measuring success in English" (Wood 1968). 

Not only are tests inadequate, criteria by which evalua- 
tions are made are evidently highly variable (Smith 1969)* 
Such data not only point to the difficulty of objective 
testing with language, they suggest that existing tests 
may not be registering significant features of language 
response. From our own survey of testing materials we felt 
that they tended to emphasize superficial competencies at 
the expense of linguistic ability. This tendency, along 
with the difficulty of language measurement in general, may 
account for. the but partial success of standardized tests 
in English . 

What standardized tests lacked, we observed, were items 
which registered a "deep" response to language in the same 
sense that the linguist distinguishes between "deep" and 
"surface" structure. This "deep" response might include 
sensitivity to underlying grammatical structures, influence 
of speaker and situation on choice of words and structures, 
nuances of meaning, and subtle relationships operating 
beneath the surface of extended discourse. Though subjec- 
tive in nature, responses to such factors needed to be 
isolated and examined to see whether or not they were mea- 
surable. This project was a beginning step in that process, 
with the specific aim of subjecting a number of likely 
responses— chiefly the ability to respond to underlying 
grammatical relationships — to a testing procedure which would 
indicate the probability of their being measured, given 
further work and more developed instruments than we would be 
able to use. 



METHODS AND PROCEDURES 



The project's principal investigator was Robert I. 
Williams, a teacher of writing and literature specializing 
in Renaissance comedy and stylistic analysis; its co- invest- 
igator was James R. Nattinger, a linguist specializing in 
bilingualism and sociolinguistics; its consultant was Morris 
Weitman, a psychologist specializing in tests and testing 
procedures. The project took from June of 1971 to June of 
1972 not counting time needed for writing of this report. 

It consisted of selection of criteria to be tested, creation 
of a testing instrument, application of the instrument to 
a criterion group and to a subject population of junior and 
senior college students enrolled in 6 sections of Writing 
323 (Junior Composition) in the academic year 1971-1972, 
and, finally, compilation and interpretation of results. 

While the results of the project were more indicative than 
final— we could not develop a fully reliable test, nor 
could we apply what we did develop to a large population — 
we succeeded in isolating 6 significant criteria of respon- 
siveness which have some probability of yielding measurable 
results in future, more extensive projects. Further, the 
techniques we used may be of value to anyone continuing 
this line of research. 

The first step in the project was selection of potential 
criteria for analysis and testing. We decided that we would 
have to be guided by two general principles; that responsive- 
ness, whatever it may be, would involve a somewhat heightened 
sensitivity to the deep factors operating in semantic meaning, 
structure of word groups and sentences, and in extended 
discourse; and second, that our choice of components would 
be limited, necessarily, by testing. Whether or not the 
final resultant group of measurable criteria were the most 
significant as indicators of responsiveness would need to 
await a considerable advance of general knowledge of lingui- 
stics, jsycholinguistics, sociolinguistics, and of what spec- 
ific capacities are likely to make one person a good poet, 
another no poet at all, or one person facile with words, 
another not. We composed rough test questions, then, on a 
number of specific aspects of the three general areas, 
semantic meaning, structure of word group and sentence, and 
extended discourse, emphasizing choices that would reveal 
a response to less than obvious factors operating in a 
particular case. James Nattinger devised the items on 
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structure of word groups and sentences, Hobert Williams 
devised those on semantic meaning and extended discourse) 
each read and criticised each other's items, rejecting 
those that were unclear or trivial. 

Beginning with a stock of rough items totalling 1 35 * 
we turned to the three problems they presented: were they 
significant, were they measurable, and were they susceptible 
to analysis? Whether or not an item was significant, both 
in itself as a meaningful choice and in respect to the ling- 
uistic feature it was meant to test, was determined by a 
group of five experts in the fields of English writing and 
linguistics. V/e chose people with experience in more than 
one phase of language study and teaching, who were acquain- 
ted with the concept of deep structure, and who were, other- 
wise, of varied backgrounds and interests: 



Gregory F. Goekjian, Asst. Prof. English, Ph.u. Pittsburgh, 

1 9?0 i specialist in literary criticism, 
poetry . 

James R . Nattinger, Asst. Prof. English, rh.D. Michigan, 

1970} linguist, specializing in bi- 
lingualism and sociolinguistics. 

Marjorie B. Nelson, Assoc. Prof. English, to. A. Oregon, 1956; 

specialist in Milton, English teacher 
education} head of reading, English 
section, E.T.S., Princeton, N.J. 



Shelley C. Reece, Asst. Prof. English, Rh.D. Nebraska, 

1966) head of Composition Program, PSU, 
specialist in composition, teaching of 
writing, rhetoric. 



Robert I. Williams, Assoc. Prof. English, Ph.D. California 

(Berkeley), 196?; specialist in 
Renaissance comedy, stylistic analysis. 



Baxter D. Wilson, Assoc. Prof. English, i-h.D. Virginia, 

1952) linguist, specializing in phonetics, 
Old English. 



Note: Nattinger and ’Williams read each other's test items 

and factored them independently, jointly making up 
the fifth member of the group. 



Agreement of 4 out of 3 cf the group was required to 
establish that a potential test item wan intelligible and 
significant. Along with agreement on significance, we 
required the group of experts to have a similar community 
of opinion on each item's linguistic components, that is, 
we wanted to assure that each item could be analyzed, either 
as a choice that would measure a specific component (e.g., 
semantic meaning) or that would measure two or more compo- 
nents interacting together (e.g., semantic meaning, word 
form, structural feature). This procedure was necessary, 
we felt, because a given language choice often has more 
than one linguistic factor operating in it, and if it were 
possible to correlate responses we might, find that a subject 
tended to be strong in choices involving, say, structure of 
word groups and sentences, and weak in those involving 
semantic meaning. 

Agreement among the experts on the linguistic compo- 
nents of each test item was achieved by use of sheets which 
had a number of "factors" which could enter into the choice 
a test item called for. Though these factors were somewhat 
crude, they were sufficiently detailed to indicate whether 
or not a given test item's response could be analyzed and 
what linguistic components, specifically, the choice invol- 
ved. (For a sample factoring sheet, see Appendix A). 

After discussing these components with the group, and pro- 
viding them with a glossary so that everyone was clear as to 
what was meant by the headings and sub-headings on the sheet, 
Robert Williams and James Natvinger joined the group as 
experts analyzing each other's test items. Each potential 
item, then, of the original collection of 135 was subjected 
to independent analysis by the group. Each of the five 
experts was a 3 ked to distribute 1 C points for each item 
among the various components he felt were operant in the 
choice the item called for. Thus the test item (l-C), which 
asked the subject to fill in a blank by choosing between 
"a real persuasive" and "a really persuasive," had the 
following distribution of factors according to one expert; 

/? social inguistic Context 

3 Semantic Conte.. 

2 Word croups 

1 Position, Positional Emphasis 

2 Underlying Grammatical Relationships 






The numbers- -parts of 10 distributed--- indicate this 
expert^ opinion of the linguistic factors operating in this 
particular language choice and their relative weights. Such 
a wide distribution, of factors was, however, unusual) typically, 
the experts saw 2 or 3 factors operating in choices,, e.g., 
an expert saw in a choice having to do with word -compounding 
(18-A) linguistic facoors of the relative weights of ** for 
Semantic Context, 6 for Underlying Grammatical Relationships. 
Where there was not substantial agreement among the experts 
on the components involved the item was excluded from the 
test. 80% or more agreement was required. 3y this process 
we arrived at a collection of 28 test items, many with two 
or more parts, totaling 7^ single-choice decisions. These 
made up our test (set* Appendix B). 

As might have been anticipated, the testing instrument 
did not have items evenly distributed among choices involving 
semantic meaning, structure of word groups and sentences, and 
extended discourse i the largest part of the test was devoted 
to items on structure, with the remainder devoted to semantic 
meaning and extended discourse. Further, test items would be 
shown to be invalid because of vague wording and the like. 
However we proceeded with the testing, which called for 
giving the test first to a criterion group of 21 people 
who are acknowledged experienced users of language in their 
vocations or in their daily work lives. The . group ranged 
from students to professional writers, from secretaries to 
writing teachers. We specifically avoided a predominance 
of college teachers of English because we felt their responses 
might reflect only one dialect of our language, written standard 
English, and, further, because many had had training which 
would allow them to see deep structure as an intellectual 
concept, thus clouding the intuitive response the test was 
designed to ellicit. The responses of the criterion group 
became the basis for judging "right" and “wrong" answers in 
the main group of subjects, 158 students enrolled in our 
junior-level writing classes. Because of lack of agreement 
by members of the criterion group our stock of items was to 
be further reduced at the time of analysis. The standard 
applied was that lb responses out of the 21 of the group 
should be in agreement on any specific item. Disagreement 
appeared to come fr ore ambiguities \n the instrument, some 
confusion over deep structure (a concept unfamiliar to most 
of the group), and variability of responses with items that 
attempted to reach overly fine discriminations. Requiring, 
then, the agreement of lo out of 21 for each item, the test 
war? finally reduced from 73 to 52 items for purposes of 
analysis. Meanwhile, & / 3 item test was given to the 
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experimental group, composed of junior-uenior students in 
Writing 323* 

The six writing classes were taught, three each, by 
Williams and Hettinger, with parallel texts, parallel readings, 
parallel paper assignments and exercises, and, in so far as 
was possible, the same styles of classroom instruction. Tne 
chief purpose of instruction, in this case, was clarification 
of the idea of deep structure arid establishment of some 
common store of knowledge about language in general. The 
last was most important because we had no way of knowing 
what the background of students would be. As it turned out, 
we had little difficulty in explaining the notion of deep 
structure, apparently, and we proceeded in each class to 
administer a pre- and a post-test. The pre-test was given 
the middle of the seoord week of instruction, the post-test 
was given at final examination time. In both testings, 
students were advised that no grade was attached to results, 
and, except for the fact of there being two testings, were 
informed what the project was about. Actual beginning enroll- 
ment in the courses totalled 246, but dropping of a class, 
failure to take both the pre- and post-test, and other causes 
resulted in a decreased subject population, 158* While this 
reduction was more than we had hoped would occur, the limited 
scope . of the project made its impact less serious than it 
would be in a fully developed study. 



RESULTS 



The overall results of analysis of testing point toward 
six areas of measurable responsiveness » response to (1) 
sociolinguistic context, (2) semantic contexti v3) deep 
structure of phrases* (4) deep structure of sentences, 

(5) primary semantic factors of extended discourse, and 

(6) organization of discourse, 

Sociolinguistic context refers to the social setting 
in which language occurs, and involves the nature of a 
speaker and similar factors i it is not to be equated with 
"propriety," since many text items ranged over different 
levels and styles of expression and involved formal as well 
as informal social situations, semantic context means, more 
narrowly, the connections with and influences on a given 
word or expression by surrounding words and meanings, it 
differs from "semantic meaning" (in the sense of "dictionary 
definition") by emphasis on the modifications of a word or 
expression brought about by the linguistic surrounding in 
which it occurs. 

The two areas of deep structure are merely different 
ranges of the same phenomenon, and their meanings as we use 
them refer to the fundamental concept of deep structure as 
distinct from surface structure. Items were designed to 
test a subject's response to complex connections underlying 
phrase and sentence formulation, connections which go deeper . 
than "agreement between subject and verb" or other superficia 
arrangements we observe as users of the language, ihe 
distinction, then, between the two areas is mainly one oi 
the size of the linguistic unit, i«e«, phrase or woid-group, 
and sentence. 



primary semantic factors of extended discourse (listed 
on our factoring sheots as "Primary Factors of Extended 
Discussion," the last word being a misreading for "discourse",/ 
refers to the basic semantic component of discourse, discourse 
being any unit of verbal communication larger than the 
sentence. As with semantic context primary semantic factors 
involves more than the "dictionary definition" of a word j 
it includes the basic meaning of a word and the modifications 
of that meaning through the influence of surrounding linguistic 
elements. This individual and contextual meaning becomes, 
in extended discourse, further modified oy structure of sentence 
and of paragraphs, but we treat it as a distinguishable element 



for purposes of analysis* The same may be said of organiza- 
tion of discourse, which is, absolutely speaking, tied to 
the meanings of words, word groups, sentence form, and devices 
such as repetition, but which we treat as a separate factor. 
Organization of discourse includes such things as compari- 
son-contrast, spacial arrangement, temporal arrangement, 
analogy, induction, deduction, cause and effect, and syllo- 
gistic form (See Appendix A). 



These six areas are those which, out of a number we 
built into the test, survived analysis by experts and testing 
with a criterion and experimental group, and which emerged 
from analysis of results. Of the 73 items of the testing 
instrument (reduced by some disagreement among the criterion 
group) 52 items qualified for analysis and yielded the 
following distribution. 



Nature of item 



Number of items 



Sociolinguistic Context ** 

Semantic Context J 

Deep Structure, word-group ? 

Deep Structure, sentence 12 

Primary Semantic Factors of Extended 

Discourse 8 

Organization of Extended Discourse 8 

Number of items 52 



Because three test items involved substantially more 
than one linguistic factor, the total number of items fed 
into computor analysis was 55* These six areas of apparently 
valid linguistic choice became t in effect » criteria of 
responsiveness the project was designed to search out. 

However, the question of their reliability and validity 
remained, and further analysis was conducted to see to what 
extent items on the test registered differences of response 
and what these differences might reveal, in order to study 
reliability and validity, six scales were constructed, one 
each for the factors mentioned above. Since the attributes 
in question are dependent variables which presumably are 
amenable to change over time, test-reteot reliability is 
inappropriate. Consequently pre-test and post-test data 
were analyzed separately, providing a form of replication. 
Odd-even split-half correlations were computed for each scale 
and corrected for reduction in length by the Spearman-Brown 
prophecy Formula. As a further check on attenuation produced 



9 . 



by insufficient number of items, Scales A and B were combined 
(odd items from A plus even items from B versus even items 
from A plus odd items from B), Scales C and D were combined 
(same as A and B), and Scales E and F were combined (same as 
A and B). 

Factor A, Sociolinguistic Context, consists of eleven 
items which have score weights ranging from l6 to 35 with a 
median of 28. Reliability estimates are a little low (.55 
and . 52 ) but could well be raised by further development. 

Factor B, Semantic Context, consists of seven items having 
score weights ranging from 12 to 32 with a median of 28. 
Reliability estimates appear to be quite low (.14 and .21); 
this scale does not look very promising. Combining A and 
B yielded somewhat higher reliability coefficients (.67 and 
. 61 ) but the increase is not as great as would be obtained 
by adding seven items to Scale A. 

Factor C, Deep Structure (word-group), consists of nine 
items having score weights ranging from 29 to 42 with a 
median of 37. Reliability coefficients are about the same 
as for Factor A (*56 and • 53 ) • 

Factor D, Deep Structure (sentence), consists of twelve 
items having score weights ranging from 27 to 43 with a 
median of 38 . Reliability estimates are at the same level 
as for Factors A and C (.56 and .55) • Combining Scales 
C and D has relatively little effect on reliability (.63 
and .63). 

Factor E, Primary Semantic Factors of Extended Discourse, 
consists of eight items having scoring weights ranging from 
21 to 46 with a median of 30» Reliability is slightly lower 
than for Factors A, C and D (»5D and .47)* 

Factor F, Organization of Extended Discourse, consists 
of eight items having score weights ranging from 12 to 45; 
the item weights are 12, 40, 42, 45, 45, 45, 45 , 45 . 
Reliability coefficients are quite low (.05 and..3l)» 
Combining E and F produces reliability coefficients of 
.32 and .54. 

Reviewing the reliability findings (Table 1 ) it appears 
that there is no advantage to combining scales (A and B, 

C and D, E and F). It also appears that Scales A, C, D 
and E are promising whereas Scales B and F seem less worth- 
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Performance on each of the six factor scales was analyzed 
separately by means of classes nested in Instructors ANOVAk 
with two repeated measures (pre-post). 

Since this study represents rather crude attempts to 
develop measures of some constructs derived from a particular 
theoretical approach to language, refined validation studies 
are rather premature. But it was considered useful to carry 
out some studies of this sort as a way of getting some hints 
as to the probable worth of the measures which had been 
developed. As displayed in Tables 3 through 8, it was found 
that there were student-gains in performance on Factors A, 

B, C and D by the end of the writing course and no pre-post 
change in performance on Scales £ and F. Though these 
changes are as predicted, they are quite small in magnitude 
(Table 2). It is quite possible that this is partly a 
reflection of the insensitivity of the instrument indicated 
by the less than satisfactory reliabilities found for the 
six factor scales. 
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Table 1 



Reliability* of Factors and Testing Occasion 
Factor Pre-Test Peat-Test 



A 


.55 


.52 


B 


.lU 


.21 


& B 


.67 


• 6l 


C 


.56 


.53 


D 


.56 


• 55 


& D 


.63 


.63 


E 


.50 


.1*7 


F 


.05 


.31 


& F 


.32 


.5h 



*Odd— even product— moment correlation corrected with Spearman— Brown 
Prophecy Formula; all 158 S's participated on both testing occasions. 
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Table 2 



Mean Factor Scores 



FACTOR 





A 


B 


C 


D 


E 


F 


Instructor 1 


215" 


132 


250* 


W* 


191 


23§" 


Instructor 2 


225 


133 


271* 


366* 


200 


235 


Class 1 


20 k 


131 


255 


356 


196 


238 


Class 2 


205 


130 


258 


332 


177 


235 


Class 3 


223 


126 


256 


355 


188 


251 


Class 5 


22k 


138 


250 


355 


201 


231 


Class 5 


22k 


135 


283 


375 


200 


228 


Class 6 


225 


135 


277 


373 


203 


237 


Class 7 


235 


135 


262 


360 


203 


220 


Class 8 


212 


130 


265 


356 


192 


259 


Pre-Test 


215* 


129* 


255* 


355* 


197 


237 


Post-Test 


. 222* 


135* 


265* 


359* 


192 


233 



C. 



Factor A Pre 
Post 

Factor B Pre 
Post 

Factor C Pre 
Post 

Factor D Pre 
Post 



CLASS 



1 


2 


3 


5 


_L_ 


6 


8 


. _8_ 


195" 


192 


227 


230 


217 


220 


227 


212 


215 


218 


219 


218 


231 


230 


251 


212 


132 


126 


120 


131 


133 


133 


123 


150 


130 


135 


133 


156 


138 


137 





120 


255 


250 


220 


255 


281 


273 


255 


263 


256 


256 


272 


255 


285 


281 


269 


265 


342 


328 


316 


329 


"1ST 


371 


357 


357 


369 


337 


350 


359 


381 


375 


363 


359 







Table 3 



FACTOR A ANOVAR 



Source of Variation 


SS 


df 


MS 


F ratio 


si& 


Instructor 


6880 


1 


6880 


1.91 


ns 


Class (Instructor) 


23928 


7 


3418 


0.95 


ns 


Error (Between) 


535678 


149 


3595 






Total (Between) 


566486 


157 








Pre-Pos t 


4221 


1 


4221 


4.26 


• 01 


Instructor x Pre-Post 


229 


1 


229 


0.23 


ns 


Error (Within) 


147590 


149 


991 






Total (Within) 


172659 


158 
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Table 4 



FACTOR B ANOVAR 



Source of Variation 


SS 


df 


MS 


F ratio 


Sig 


Instructor 


280 


1 


280 


0.24 


ns 


Class (Instructor) 


4237 


7 


605 


0.52 


ns 


Error (Between) 


172439 


149 


1157 






Total (Between) 


176956 


157 









Pre-Post 


2898 


1 


2898 


7.51 


.01 


Instructor x Pre-Post 


957 


1 


957 


2.48 


ns 


Class (Instructor) x Pre-Post 


10877 


7 


1554 


4.03 


.01 


Error (Within) 


57570 


149 


386 






Total (Within) 


72302 


158 









O 

ERLC 
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Table 5 



FACTOR C 

Source of Variation 
Instructor 35056 

Class (Instructor) 11608 

Error (Between) 838224 

Total (Between) 884888 

Pre-Post 9934 

Instructor x Pre-Post 1246 

Class (Instructor) x Pre-Post 57998 
Error (Within) 225300 

Total (Within) 294478 



ANOVAR 



df 


MS 


F ratio 




1 


35056 


6.23 


.01 


7 


1658 


0.29 


ns 


149 


5626 






157 









1 


9934 


6.57 


.01 


1 


1246 


0.82 


ns 


7 


8285 


5.48 


.01 


149 


1512 






158 
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Table 6 



FACTOR D ANOVAR 



Source of Variation 


SS 


df 


MS 


F Ratio 


Si£ 

.01 


Instructor 


51160 


1 


51160 


8.35 


Class (Instructor) 


28692 


7 


4099 


0.67 


ns 


Error (Between) 


913146 


149 


6128 






Total (Between) 


992998 


157 









Pra-Post 


17972 


1 


17972 


11.51 


.01 


Instructor x Pre-Post 


6684 


1 


6684 


4.28 


.05 


Class (Instructor) x Pre-Post 


61540 


7 


8791 


5.63 


.01 


Error (Within) 


232652 


149 


1561 






Total (Within) 


318848 


158 
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Table 7 



FACTOR E 






AN0VAR 






Source of Variation 


SS 


df 


MS 


F ratio 


SiS. 


Instructor 


5842 


1 


5842 


2.18 


ns 


Class (Instructor) 


16890 


7 


2413 


0.90 


ns 


Error (Between)* 


400016 


149 


2685 






Total (Between) 


422748 


157 









Pre-Post 


1969 


1 


1969 


1.87 


ns 


Instructor x Pre-Post 


885 


1 


885 


0.84 


ns 


Class (Instructor) x Pre-Post 


10245 


7 


1464 


1.39 


ns 


Error (Within)* 


157268 


149 


1055 






Total (Within) 


170367 


158 









* Winer C1962) recommends pooling nested factor sums of squares with appropriate 
error terms when none of the nested factors reaches significance at the .10 
level. This is of course a matter of controversy and in this particular analysis 
would make no difference in outcome. 
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Table 8 



FACTOR F 



ANOVAR 



Source of Variation 


SS 


Instructor 


124 


Class (Instructor) 


19336 


Error (Between)* 


585660 


Total (Between) 


605120 



Pre-Post 


1164 


Instructor x Pre-Post 


80 


Class (Instructor) x Pre-Post 


9964 


Error (Within)* 


347554 


Total (Within) 





df 


MS 


F ratio 


Slg 


1 


124 


0.03 


ns 


7 


2762 


0.70 


ns 


149 


3931 






157 









i 


1164 


0.50 


ns 


i 


80 


0.03 


ns 


7 


1423 


0.61 


ns 


149 


2333 







* Winer (1962) recommends pooling nested factor sums of squares with appropriate 
error terms when none of the nested factors reaches significance at the .10 level. 
This Is of course a matter of controversy and In this particular analysis would 
make no difference In the outcome. 



CONCLUSION 



Taken together, the results of the reliability and validity 
studies indicate that scales A,B,C,D, and E merit further develop- 
ment. Discarding scale F may be premature since it may have been 
inappropriately employed and may prove worthwhile if tested on 
freshmen and sophomores rather than on juniors. These scales, 
and the criteria they represent, relate to individual items in 
the testing instrument given in Appendix B in the following manner. 



Scale and Criterion of Responsiveness Test Items Used 

in Final Analysis 



1. 

2 . 

3. 

4. 

5. 

6 . 



Factor A, Sociolinguistic Context 

Factor B, Semantic Context 

Factor C, Deep Structure (word-group) 



Factor D, 



Deep Structure (sentence) 



Factor E, Primary Semantic Factors 
of Extended Discourse 

Factor F, Organization of Extended 
Discourse 



la , lb , lc , 2a ,2b, 

2c , 3a , 3b , » 5b , 5c • 



3a , 3c , 5b , 6a , 7a , 
13c. 

I6a,l6b,l6c,l7a, 
I7b,l7c,l6a, 18b, 
18 c. 

I0a,l0b,llb,l2a, 
12b, 12c, 13a, 13b, 
I4a,i4b,l5a,l5b. 

I9a,l9b,20a,20b, 
23a, 24a, 24b, 24c. 

22a, 24c, 25 a, 27a, 
27b, 27 c, 28a, 28c. 



For those interested in pursuing these criteria of responsive- 
ness to English, actual test items should be consulted. 



All in all, the researchers felt that a crude but indicative 
beginning has been made to define and measure aspects of language 
response not ordinarily considered and perhaps not even measured 
by standard instruments. It is hoped the criteria of responsiveness 
we have located will be further refined in the development of more 
sophisticated language tests than those now available. 
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APPENDIX A 
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odOocoU 



an 



Factor* 



Soclollngulatle Context 
(Register) 



Semantic Context 



Connotat ion-Denotation 



Formal Signals Influenc- 
lng Semantics 



Other (Specify) 



Formal Signals (prep's* 
con.1 1 a 2 



Word Groups 



Complete-Incomplete Stmt. *| 2 
Position-Position gmp haslsLct** 

' J I s 

J 9 O 



joinery or wora Groups* 
S - Verb 



Sentence Type-Form 
Other (Specify) 



Underlying Gramma t’l 
Relafahlp 



} 



0 
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Co 

ct 

1 

ct 



CO 
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CtXS 



I 
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CO 



Relations hip Between Sent. 



Primary Factors of Ext»d 
Dlaouaalon 





Comnarl ao n-Gontraat 

Special Arrangement 

Temporal Arrangement 

Association 

Analogy 

Induction 

Deduction 

Cause-Effect 

Syllofla tle Form 

Other (Spe elfyi 
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H>D 

(A H» 
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o at 

ct 






10 O 
• D 
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oo 
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“•oS 

g: 

CO 
00 



Larger Forms of Discourse 

(Par. ) 

Rhetorical Form 




* J 




ITEM # 

PARTS 7 GENERAL DESCRIPTION OF ITEM . SUBJECT 



APPENDIX B 



LANGUAGE PREFERENCE TEST 



The following test is designed to find out how your responses to the 
English language compare with those of people who have acknowledged 
sensitivity to its nuances. The test goes on no official record, nor 
does its ultimate result influence your course grade. 



The majority of items you are asked to respond to pertain to written 
English. Every item or parts of it asks you to choose between two 
alternatives; every item must be marked, and an item not marked will be 
counted as '’wrong!* The test is timed. Items not completed will have 
to be counted as ’wrong," so the most advisable procedure is to mark down 
what you instinctively feel is "right" of a particular case, rather than 
marking down a choice after long deliberation. 



For purposes of linguistic study you are asked to fill in the infor- 
mation requested below. This information, along with your name and score, 
will, be kept with the test for a month; after that tine your test will be 
ass * : sued a number and your name and information about you destroy • ® 8 

results can be found out by consultation with the instructor during that 

mouth . 



Name . 

Writing 323, Sec. 

Sex 

Home address (family) 

Other languages spoken at home 

To what extent? 

Previous formal instruction in English language structure 



Current year in college 

Special notes, if any, on the above: 
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From the alternatives below choose the one which fills the blank so as to 
provide continuity of idea in the following passage. 
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The writer wants to delete some repetitive words (or terms) in the passage 
below. Of those underlined which are most repetitive and most likely to come 



And I think you can see the reasonin' behind that. 



w > 
i T 

N> Is) 
*• 

? 



ttl 

I 



? 

to 



■r 



ro H> 



s’ 

e 



0 

a 

o 

5 

"0 

H 

H* 

O 

0 

CD 

O 

O 

CD 



CD 

B 

a. 

O 

8 

CD 

>a 

& 

§ 



*i 

CD 






to 



CD O 

D) H* 



C D 
cr c- 

CD 

0 O 

* S 

CD 

*0 

a h 

•S f 

- CD 

co 

85 8 

65 8 
a*o 

o CD 

* S 

§ c 

3* 

o g 
. o* 

La* 

O O 
H* C 
0 H 

, 

o *n 
|c C 



. *0 

r> 

ft 

0 



H 

CD 



8* 



O « 
CD CD 
CO 

CD *0 
H» H 
0 H* 

oq a 

CD 

H* 

0 O 

a c 

6 3 

ft CD 

3 



3 

CD 



CD 

tt 

O 

0 



0 *0 
P 0 
H* H 

0 ft 
CO 

CO o 



[< CD ** ^ 

He cl 



O 

c 

H 

CD 

O 

o 

0 

o 

* 



o 

0 



o 

o 

a 

*0 

H 

0 

1 



Is) 



O 

0 

CD 



ft 

nr 

CD 



a* 

CD 

■3 

H* 

3 



o cr 


0 

OQ 


Hi 0 


0 


H* 


ft 0 


o 


ctoq 

0 

0 


Hi 


0 


s < 


c 

H 


CL H* 


0 ft 


0 


•3 a 


O 

o 



0 

o 

41 



O Hi 

3 3 

ft 

cr 
ft CD 
O H* 

V< CD 
O « 

C 

• o 
e 



CD 

n 



8 

CL 

0 

CD 



O 

M 

C 



CD 

CD 



O 

S 

cr 

CD 

h 

CD 

*0 

M 

8 

CD 

Cl 



cr 

o 

e 



H* *0 ft < 

rr h cr h 

H* CD rt 

o 



0* CD 
CD CO 
0 

< o 



P 

§ H 
CL*0 
0 0 
H H 



CD^-0 ft 

e 0 g o 

0 ft 5 M* 
H* H* 

* ft 0 
H»CO 
r < co 
8> fl> ^ h 

S O H» 
CD 0 Hi O 

Q 0 0 

O CL o - 
c CO 

Con 
e hi 
h 0 o 
ft no 
0 0 o a* 
*n 0 

rr e o *0 
o hi n 
*0^0 

• s 

0 

H* 0 
O H* 
n C 0 
0 ^ OQ 
h- t 
H* H* H* 
0 h- 0 
cthu 

H» ft 



s 



H- 3 

0 ft 



II 



0 Hi i 



CL'S 

c o 
n 0 
0 



^ MO ft O 2 

o O H» H* C O 

g s H C o ft H 

n n a rt 0 0 

_ h* o *n 0 

D 0*0 P*tt C H 

3 C r (D ft hH 

o 55 0 h n v<j 

CL H* V* • 0 7T 

C 0 O ft H O 

Q 0 c 0 O VC c 

ft ® m cr • h 

H* 0 n H» 0 

O'* 0 0 C 

0 So :c p 

^ g> 0 H* n 

*0 0 0 0 rt h 

H* CL 0 _ TJ cr 0 

ft ^ D 0*0 0 

ft e io hHit 

c 0 ft 0 0 v< 

h cr h rt n 

id c io (o * w a 

• O *0 0 0 

*0 H H» 0 

0 0 0 Hi O CO 

g 0 o 0 h 

0 ft 0 0 0 CL 0 

_ n* g o* _0 o 

*0 0 0 _ cr H rt 

h rr h* < o 0 * 

H* 0 H* I— 1 * O 

CL > 00 rt rt O 

0 5 nr m 0 - < 

tO CT O » Q 0 

O O CD 0 2 

c o cr rt o 0 

H H* c < *0 

0 0 0 O ft 0 0 

0 *Q Ou o ft (i ti 

M C CO 0 * rt 

1 n ft ft i 0 

0 *0 P O 

» 3 (o ft o*c ^ 

10 ft o HH J 

O 0 • 0 H* 

0 rr v<- cl O o 

_ ^ O 0 H & 

cr to 5 c » oq 

0 e 

n 0 

< w H» 0 

fc& S’ 1 

CL | 



0 ft 0 
H» 0 
0 ^ 

OL 0 

0 H* C 

i 0 0 



cr o 

0 hh 



3 



O OQ 



£ 



0 

1 



o 



ERLC 



47 . 





W 




> 




• 




• 


ro 


H 1 


N3 


M 


• 


• 


• 


• 




S3 


CO 

e 


£ 


< 


ft V} 


to 


O 


ft 


CO 


•a 


c 


M 


< H- 


ft 


0 


ft 


ft O 


p 


c 


ft 


H» ft 


5 


p 


H* 


£* H» 


ft 


ft 


Q 


ft 




rt 


3 


^ CL 




H* 




• ft 


ft 


O 


o 


rr 


ft 


0 


Hi 


£ 


H* 


o 


ft 


w* 


0 


Hi 


G, 


ft 


a 


a 4 


CL 




ft 


ft 


ft 


a 


co 


H 


CO 


ft 


o 


ft 


o 


rt 


H 


O 


H 


ft 


H* 


ft 


H- 


o 


-o 


ft 


or 


rr 


rr 


H 


ft 


H- 


H* 


— 


CL 


<* 


< 


ft 


rt 


ft 


ft 


e* 


o 


ft 


a 


B 




rr 


ft 


*9 


00 


O 


rr 


H 

ft 


< 


v3 


ft 

H* 


ft 

ft 


ft 


w 


to 


H- 


C 




O 


K 






a 








co 


ft 






o 


CO 






Hi 


g 






rr 


8 






a 4 


ft 






ft 


o 






< 


rh 






ft 










rt 






M 


*-» 






ft 


ft 












• 


0Q 








ft 








O 








00 








M 








ft 
















ST 
















o 








Hi 















H* 


ft 










n 


a 










a 4 




H* 


n or 


Hi Q. 


M 


IT 

o a 4 


0 


a* ft 


H ft 


0 


ft 


ft 




ft 00 


O Hi 




CO 




rr 


0 ft 


0 H- 


H> 


rt *a 


P 4 


_ 0 


- ft 


H 




co 


ft 


o 4 


g, 5 


O 


a 


ft 




ft rr 


a 4 rt 


3 


ft 


co 


a 


o o 


ft H* 


rt 


CO 


ft 


H* 


?r 


rt 




n 


00 


ft 


o 


<# 


M 


H 


ft 


rr 


rr c 


ft 


ft 


H* 




6 § "S S* 


O' 

ft 


cr 

ft 


O 


ft rr 


ft n 


rt 




M 


ft 


h sr 


ft 7r a 4 


* 


o 


• 


a ft 


H H* 


ft 


a 4 


K 




co 


ft 0 




ft 






< 


CL OQ 


<0 


rt 


ft 




n ft 




ft 




5 




sr m 


rr rt 


H» 


rt 


CL 




ft M 


o a 4 


H 1 


a 4 






ft 


ft 


ft 


ft 


rr 




B ^ 


O 






*■** 




o 


ft M 


« 


*o 


ft 




C Hi 


ft 




ft 


a 




5 H* 


ft 0 




0) 






rr o 


„ P- 


> 


CO 


o 




ft o 




0 


ft 


a 4 




H* H 


O H- 


CL 


oo 


o 




0 • 


h* a 




ft 


o 






r> 


ft 




CO 






ft CO 


n 


ft 


ft 




H* 


0 rt 


H 


a 






Hi 3 


H» H- 


o 


•a 


ft 




H* CL 


ft Hi 


CO 


a 4 


& 




0 H* 




CO 


ft 


o 




ft 0 


Hi H- 




ft 


a 




M 00 


H* 0 


H* 


H* 00 




M 


co oo 


rt 


N 






^ ft 


CO 


VO 


ft 


rr 




• a 4 


c a 4 




CO 


a 4 




H 


H ft 


rt 


• 


ft 




ft O 


n> ft 


or 








M C 


» rr 


ft 




•o 




9 02. 


• 






ft 




5 p 4 


ft 






H* 




H* 




3 




H 




^ n 


ft- i-3 


o 




CO 




« a 4 


H O 


c 








ft 


H* 


9 




o 




a 


ft ft 


rt 




Hi 




H- o 


a a 4 


ft 








co ft 


ft 


H* 




ft 




ft a 


o 


0 




h- 




*0 rr 


H M 


CO 




rt 




*0 ft 


ft ft 






ft 




ft H 


ft Hi 


CO 




H 




ft 


K rt 


rt 




a 




H O 




O 




ft 




H* Hi 


O 


O 




n 




0 


ft 


a. 




H* 




00 rt 


c 






<: 




sr 








ft 




ft 








CO 




M 








rr 




ft 








sr 




0 








ft 




ft- 




















o 












a 












ft 












CO 



48 . 



O 

ERIC 



SO 




From the alternatives given below choose one of eaci pair which best describes 
how the given passage is organized. 




From the pairs of alternatives given, choose the one vhich best describes the 
way the passage below is put together. 
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each of the alternatives below choose the one of each pair which is isost 



